Field of the Invention
The present invention relates to an image processing apparatus, an image processing system, a control method for the image processing apparatus, and a non-transitory computer-readable storage medium and, more particularly, to a virtual reality technology and mixed reality technology which use an HMD (Head Mounted Display).
Description of the Related Art
In recent years, as a technique for seamlessly merging a real world and virtual world in real time, so-called MR (Mixed Reality) techniques are known. As one of the MR techniques, a technique which uses a video see-through HMD (Head Mounted Display) is known. This is a system in which a video camera or the like captures an object, that nearly coincides with that viewed from a pupil position of an HMD user, and the HMD user can view an image obtained by superimposing CG (Computer Graphics) on the captured image.
This system is formed by an HMD for capturing an external world and displaying the image to the user, and an image processing apparatus for generating an image by superimposing CG on the captured image. To experience an MR space using the HMD while freely moving, the HMD and an external apparatus such as a PC preferably perform communication by wireless transmission. In general, however, the communication band of wireless communication is narrower than that of wired communication, and wireless communication is thus unstable.
To solve this problem, Japanese Patent Laid-Open No. 2004-120201 discloses a technique of enabling stable transmission of image data by changing the compressibility of an image in accordance with the status of a communication network to adjust a data amount in addition to compression and transmission of image data.
In wireless transmission between an HMD and an external apparatus, an image captured by the HMD is transmitted to the external apparatus, and the external apparatus measures a position and orientation, thereby performing image composition. In this case, to suppress the network transmission amount of image data from the HMD to the external apparatus, the image data is compressed by the HMD, and a coded stream is transmitted to the external apparatus. Upon receiving the coded stream transmitted from the HMD, the external apparatus decompresses the coded stream, performs image analysis using the decompressed image, and then measures a position and orientation based on image analysis information. Upon completion of measurement of the position and orientation, the external apparatus performs image composition using the decompressed image, and transmits the composite image to the HMD for display on the HMD.
In the technique described in Japanese Patent Laid-Open No. 2004-120201, however, when an external apparatus performs all calculation operations associated with measurement of a position and orientation, a coded stream obtained by compressing image data is used. Consequently, decompressed image data degrades as compared with image data obtained at the time of image capturing, and a calculation operation such as position and orientation measurement processing of the succeeding stage is performed using the degraded image data, thereby making it impossible to obtain sufficient calculation accuracy.
The present invention has been made in consideration of the above problem, and provides a technique of controlling data transmission while maintaining the calculation accuracy of processing of the succeeding stage.